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ABSTRACT 
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factors in learning. Three overcirching qualitative issues are 
discussed: (1) the application of traditional criteria such as 
validity, reliability, and objectivity; (2) coping with sexist biases 
in everyday and research settings; and (3) coping with apparently 
chaotic, incomprehensible findings. Five arguments in support of 
temporary understanding through qualitative research are reviewed, 
including individual differences, cr^os theory, the evolutional 
chciiTacter of development and learning, the roles of consciousness amd 
free will in human activity, and quantum mechamics theory, it is 
concluded that qualitative research is necessary to gather and 
interpret people's multiple constructions of reality. (86 references) 
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Introduction 

Philip Jackson is giving his Presidential Address at the 1990 Annual Meeting 
of the American Educational Research Association in Boston. He reminds his 
audience that the value of conclusion-oriented research (as distinct from decision- 
oriented) is that it will change "the prevailing view" when its results show that the 
researchers found their data to be complex and puzzling: 

"Their words encourage us to envision something far more complicated and 
dynamic. They speak of a restlessness emerging out of discussion. They talk 
about findings that seem to conflict, which implies the possibility of being 
mistaken. They depict investigators as being puzzled, as noticing things that 
as yet cannot be described in words, phenomena for which language is lacking. 
They speak of boundaries getting blurry, of topics merging into one another. 
Investigators, we are told, "find themselves assembling what they know in a 
new way," almost as though they were doing so unconsciously or perhaps even 
against their wills" (Jackson 1990:9). 



If we are to help change "the prevailing view" with complex, exciting and 
challenging new research, Jackson concludes, we must 

"become self-reflective about our research goals and methods and what they 
mutely and perhaps inadvertently communicate about our enterprise to the 
world at large. It requires, among other things, that we conscientiously 
explore our most deeply held assumptions and presuppositions and that we do 
so with a directness, a candour, and a tenacity that has seldom typified our 
work in the past. It was Yeats ( 1958) who said *God guard me from those 
thoughts men think/In the mind alone; He that sings a lasting song/ Thinks 
in a marrow bone' (p.236). Yeats's prayerful entreaty issued from the soul of 
a poet, yet its truth, it seems to me, applies to us all. *He that sings a lasting 
song thinks in a marrow-bone.' Those are words we all might take to heart." 
(Jackson 1990:9). 



How many "lasting songs" do we have in the distance education research 
repertoire? How many of us think in our marrow bones? Gender-exclusive language 
apart, Yeats's poetry and Jackson's reflections are relevant for us; but we have to rely 
on more than (Jod's protection if we are to challenge prevailing views. We have to 
rely on our creative and skilled use of both major research paradigms. One of those 
paradigms - the positivist, with its use of the experimental method - has long been 
predominant in the theory and practice of education. 



erJc 



3 



2 



But by the 1970's 

"Quantitative methods, relying on the hard science paradigm, had not 
delivered. As the vision (toward another paradigm) widened, qualitative 
approaches caught people's imagination (see Scriven, 1972). And so, 
qualitative research mushroomed in education" (Bogdan & Biklen 1982: 21). 



The 1970's in fact were characterized by strong debates about the merits of 
each paradigm and its methodologies - to the point where Rist had to call for the 
positivists to change their attitude from "disdain" to "detente" (Rist 1977). The effects 
of that disdain have diminished, especially among researchers who are sensitive to 
"marrow bone thinking*', who are prepared to listen and look in quiet appreciation at 
how people make meaning and act in their worlds : ''most major neopositivist 
methodologists...have shifted toward more perceptual, context-embedded,interpretive 
inquiry" (Miles & Huberman, 1990:343). The literature on this second type of inquiry 
-the naturalistic paradigm- and its associated qualitative methodologies is now 
extensive (see Appendix 1). Earlier opposition to the naturalistic/qualitative has 
given way to sophisticated discussions(eg Eisner&Peshkin 1990; Eisner 1988), 
arguments for compatibility with the positivist/quantitative paradigm (e.g., Jaeger 
1988; Howe 1987; Firestone 1987) and greater use of qualitative approaches in 
educational evaluation (Lincoln & Guba 1989; Patton 1982 ), in adult education 
(Merriam & Simpson 1989) and in disciplines other than education ( e.g., Cobb & 
Hagemaster 1987; Morgan 1983). 

But what of distance education? Can we claim much marrow-bone thinking?; 
or a balanced use of positivist and naturalistic paradigms? We cannot , but a few 
writers are arguing that we must use methodologies from each paradigm if our 
research is to mature (Hotchkis 1990; Kember et al 1990; Minnis 1985; Morgan 1984). 
Minnis has reflected, with some sadness I suspect, on the current condition of 
distance education research: 

"it is overwhelmingly descriptive, with emphasis on particular institutional 
problems... (it) reflects an a-historical and/or a-theoretical bias... tends to be 
context-specific... lacks meaningful cross-cultiu-al or comparative perspectives... 
is heavily dependent on psychological paradigms which, despite the strengths 
associated with such paradigms, tends to reduce the problems of distance 
education to ones which are amenable to the manipulation of psychological 
variables only (Coldeway & Spencer 1982; Keegan 1982; Kaye & Rumble 1981). 
What little theory exists in distance education is usually praxiological, deriving 
more from experience in the sanctioned 'practical' affairs of running programs 
than from systematic and reflective critique" (Minnis 1985: 190, 191). 
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Another researcher has deplored the "reality rape" of some statistical analyses 
and pleads for an "interpretive approach (to) give the most meaningful rendition of 
what's out there (and do) the least conceptual damage to people in their human 
interactions" (Hotchkis 1990). 

As late as 1990, staff at one well-known distance education research institute 
were outlining their research goals and methods, but all in the 
experimental/positivist traditions it appeared. No mention was made of any 
methodology apart from "experimental projects with model character" which "should 
contribute further to the scientific grounding of this know-how [about independent 
study materials] "(Baumeister, Friedrick & Mandl 1990:16, 17). This apparent 
limitation in research paradigms and methodologies contributes to what I believe are 
untested assumptions made by the authors about what they think are important 
areas to research; for example in a paragraph about the need to design study 
materials which "take over [the constant control for the teacher]", the authors state 
their goal in words which would indicate a limited awareness of the complex 
contextual factors in learning, many of which are beyond the educators control (and 
always will be ). Their words also reflect their politics and socializations: they show 
an unquestioned authoritarian view of the teacher s role which many adult and 
distance educators now find inappropriate: 

"In (face-to-face instruction), important teaching functions such as motivation, 
selection, sequencing and presentation of the teaching materials, diagnosis of 
learning response and diagnosis-based presentation of content, practice, the 
transfer of what has been learnt to new situations and other functions are 
under the constant control of a teacher who is physically present. Self-study 
materials-texts, pictures, videos, PC-learning programs and decentralized, 
multi-media learning environments based on these - must be designed in such 
a way that they can take over these functions to a large extent. The 
development and design of independent study materials therefore necessitates 
a thorough analysis of the individual processes occurring under the conditions 
of independent study: the assimilation, processing, recall and application of 
new knowledge." (Batmieister,Friedrick & Mandl 1990:16). 



There appear to be no indications that any significantly qualitative 
methodologies are being used, nor that the researchers are struggling with 
epistemological and ontological issues relating to either paradigm. 
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In reality, the use of the two paradigms are not always clearly differentiatec?, 
but competent researchers know how to use the strengths of each one to collect 
complex qualitative as well as quantitative data about a phenomenon and to 
triangulate and analyse that data without context-stripping or loss of rich description. 
For example, the research of Von Prummer and Rossie (1990) used methodologies 
from both paradigms, on the understanding that "the researchers knew the limits 
of their quantitative data, made their research values and intentions known to their 
participants, and gave them opportunities to engage in dialogue directly with the 
researchers" (Von Prummer 1990). 



Methodological Issues 

None of the qualitative methodologies for the naturalistic paradigm are soft 
options ! The methodologies have to lead toward "concrete depiction of detail, 
portrayal of process in an active moae, and attention to the perspectives of those 
studied (Patton 1980)", (Firestone 1987:20). The net results of a sound qualitative 
study illuminate complex interactions, propose new questions, challenge existing 
assimiptions, help us to appreciate how c/her people construct meaning in thoir lives, 
and expand our earlier frames of reference (Edson 1988:45). The development of such 
results demands skills, attitudes and self awareness from the researcher and detailed 
attention to academic rigour (or what Lincoln & Guba (1985) name as 
"trustworthiness"). We have to know, for example, why we should choose a 
phenomenological method to study adult learning, instead of a case study or an 
ethnography or a grounded theory study or a life history or a literature analysis. 
Each method has been discussed at length by researchers; we do not need another 
summary here. Distance education researchers can refer to writers such as Kirby & 
McKenna (1989); Strauss (1987), Taylor & Bogdan (1984) and Miles & Huberman 
(1984) for strategies. What is relevant here i'5 a review of what I consider to be three 
overarching qualitative issues, regardless of specific methods, that affect the 
maturation of distance education research. 

Now that many researchers are out of the "intellectual cul-de-sac" of 
exclusively quantitative methods and have acknowledged that "generalizations decay" 
and "statistical realities do not necessarily coincide with cultural realities" (Rist 
1982), how do they claim trustworthy qualitative data and findings when working in 
the unpredictability and complexity of natural settings? Do the traditional criteria 
of validity, reliability and objectivity still apply? How does one deal with the sexist 
biases in everyday settings? How do we cope with apparently chaotic, 
incomprehensible findings? 
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Trustworthiness has been discussed at some length (eg, Smith 1984). Lincoln 
& Guba(1985), in an excellent strategic treatment of this issue, argue that the 
naturalistic paradigm still has to answer the generic questions about a study's truth 
value, applicability, consistency and neutrality; but each aspect is covered, 
respectively, with its own criteria of credibility, transferability, dependability and 
confirmability. In order, each is the naturalistic paradigm's equivalent of validity 
(internal & external), reliability and objectivity. Clearly a qualitative researcher is 
not aiming to falsify a hypothesis nor maintain an objective, distanced relationship 
with the "subjects", nor produce results that are generalizable in the positivist 
definition: she/he is dealing with multiple realities and is not dealing with 
necessarily a truly representative sample, if ever such a sample can be found anyway! 
The qualitative findings have to provide (i) proof that his/her interpretations have 
been approved by the actors (credibility), (ii) a rich, thick description of setting and 
interaction so that the reader can be helped to judge how transferable are the 
findings to his/her own setting (transferability); (iii) evidence that instabilities and 
changes originating in the phenomena or the research process have been accounted 
for (dependability); and (iv) evidence that the data is reliable because it is 
confirmable (confirmability). Specific strategies for meeting these criteria are 
outlined in Lincoln & Cuba (1985), and some have applied those criteria into 
proposal checklists (Cobb & Hagemaster 1987; see Appendix 2). 

The issue of sexist bias affects qualitative methodologies as much as it does 
quantitative ones, but the issue generally has been treated seriously only within the 
last decade (Eichler 1988). Despite the publication of guidelines for nonsexist 
language and the existence of many scholarly articles, education research still shows 
evidence of what Eichler categorizes as four primary sources and three derived 
sources of sexist bias. The primary sources are androcentricity, overgeneralization, 
gender insensitivity and double standards. The early work of Carol Gilligan on 
developmental psychology of women (Gilligan 1982) is now a classic case of a 
researcher correcting previous research results which were not only based on 
exclusively male subjects, but also had beer generalized to include the female 
experience. Another interpretive error, for example, has been the use of a male 
oriented view of the world (patriarchy) to argue that women "subjects" were 
"deficient" because they did not conform to predictions about behaviour that were, in 
effect, based on male expectations and socializations. Belenky et al, in a ground 
breaking study about how women know themselves and their worlds (Belenky et al 
1986), point out one major limitation of research originating in Western traditions 
that attend to the male stream of experience: 

"Thus, we have learned a great deal about the development of autonomy and 
independence, abstract critical thought, and the unfolding of a morality of 
rights and justice in both men and women. We have learned less about the 
development of interdepedence, intimacy, nurturance, and contextual thought 
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(Bakan 1966; Chodorow 1978; Gilligan 1977, 1979, 1982; McMillan 1982)" 
(Belenky et al 1986:6,7). 



Even if none of these analytical mistakes are made, there i? another issue 
which relates to the researcher being the primary instrument for data collection. The 
issue is one of impact. Consciously or subconsciously, research participants invariably 
will react to a female researcher in ways different to a male researcher. Gender 
socialization is so ingrained and beyond conscious awarness for many people that it 
takes particular consciousness-raising to see it. A male researcher conducting in- 
depth interviews, for example, can never assume that female participants in the study 
will give him the same data as they would to a female researcher. 

Distance education research has to deal with this issue. We have the 
guidelines and the beginnings of serious documentation (Burge & Lenskyj 1990; 
Krkup & Von Prummer forthcoming; Faith 1988; Kirkup 1988). 

The third issue is just as fundamental, and is attracting wider scholarly 
attention. It is "the possibility that the phenomena studied in the social and 
behavioural sciences are essentially unpredictable and indeterminate" (Cziko 
1989:17). Cziko presents five convincing, indeed marrow bone level arguments to say 
that prediction and control experiments will never be successful; that we should hope 
instead for only temporary understandings. His case rests on individual differences 
of humans, chaos theory, the evolutional character of development and learning, the 
roles of consciousness and free will in human activity and quantum mechanics theory. 

Essentially chaos theory tells us (a) that it is impossible to predict an outcome 
over a progression of events, even though strict causality can operate at each single 
event in the progression; (b) that the dependence on initial conditions for the 
outcome can be so sensitive as to cause large and/or unpredictable processes for that 
outcome, and (c) that patterns and sequen-ies can be found in apparently chaotic 
conditions, if only we know how to detect them(Gleick 1987; Prigogine & Stengers 
1984). The now famous "butterfly effect" was originally developed in a meteorological 
context: the flapping of a butterfly s wings in Brazil could eventually start a tornado 
in Texas. This theory has enormous implications for research into the many inter- 
relational dimensions of teaching and learning; e.g., how do home/work life factors 
affect course learning outcomes? Can chaos theory help us better understand course 
withdrawal and completion? Can it help illuminate patterns of energy and attention 
in learning? 
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If we accept an evolutionary approach to learning (argues Cziko), i.e., that the 
random variation and natural selection applies as it does in the natural world, then 
we have to accept that: 

"Even if a previously found "effective" learning environment could be replicated 
exactly, it would be very unlikely to lead to the same learning outcomes, even 
in an identical physical environment using genetic clones of the original 
teacher and students" (Cziko 1989:20). 



The fourth Cziko argument against prediction and control approaches to 
research is based on human consciousness and free will. Our complex and creative 
behaviour cannot be known in advance - it is not a deterministic system, nor is our 
consciousness "available" to a researcher for experimentation. 

The fifth and final argument that Cziko uses to press for descriptive and 
interpretative research in education is based in quantum mechanics.Scientists have 
changed their thinking from a Newtonian view of the world as predictable and 
clocklike in its operation to a view that recognizes the random behaviour of subatomic 
phenomena. Researchers therefore have to think about probabilities in behaviour 
rather than predictabilities. Other aspects of quantum mechanics eg, a simple 
measurement of a particle can change that particle, or "that the uncertainties and 
probabilities observed in quantum phenomena are intrinsic to the phenomena 
themselves and not the result of incomplete knowledge" (Cziko 1989:23) must surely 
have relevance to the worlds of learning and teaching. 

We must apply Cziko s arguments to distance research. We must enlarge our 
boundaries of experience and exploration; use qualitative methodologies to gather and 
interpret people's multiple constructions of reality. We can show how context 
contributes to meaning whether our research participants are learners, tutors, policy 
makers, evaluators, learning designers, etc. We can "find [ourselves] assembling 
what [we] know in a new way". "Marrow bone thinking", those qualitatively oriented 
methods that will create the "lasting songs" of Yeats s plea, demand research that is 
holistic, not reductionist; cumulative, not repetitive. It also demands that we, as 
thiiiking and feeling humans involved with our research participants, be open and 
articulate about ourselves, our values and our meaning-making. 
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Checklist for Assessing a 
Proposal for Qualitative Research 



1. Expertise Yes No Cannot 



1.1 


Does the researcher demonstrate understanding of 
and experience with the naturalistic paradigm and 
the particular qualitative method/s being used? 


[J 


[J 


Saj 

[J 


1.2 


Are appropriate references cited? 


[J 


[J 


[J 


1.3 


How much experience in in-depth interviewing and 
observation is evident? 


r T 

[J 


[J 


u 


Problem and/or Research Question(s) 








2.1 


Is the lack of knowledge or problem clearly 
delineated with an appropriate rationale for 
using qualitative approaches? 


[J 


[J 


[J 


2.2 


Is an explanatory background to the problem provided? 


[J 


[J 


u 


2.3 


Is there a single, broad research question? others? 


[J 


[J 


u 


2.4 


Is the scope of the question(s) manageable within the 
time frame and context of the study? 


u 


[J 


[J 


2.5 


Is the significance of the study argued? 


[J 


[J 


[J 


Purpose 








3.1 


What is the purpose of the study?: discovery, 
description, and explanation; conceptualization 
(theory building), illustration, or sensitization? 


[J 


[J 


[J 


3.2 


Is it clearly stated? 


[J 


[J 


[J 



Modelled by E. J. Burge (1990) after Cobb, A. K. & Hagemaster, J. N. (1987). Ten criteria for evaluating 
qualitative research proposals. Journal of Nursing Education . 26(4), 138-143. 
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4. Literature Review 

4.1 Does the proposed qualitative approach call for a 





literature review and/or conceptual frsimework prior 
to initiating fieldwork? 


u 


[J 


[J 


4.2 


If so, is the review sufficiently comprehensive? 


u 


[J 


[J 


4.3 


Are major concepts identified and defined? 


[J 


u 


[J 


4.4 


If a literature review is appropriate only after data 
collection, does the researcher outline a process for 
accomplishing this? 


u 


[J 


[J 


4.5 


If bracketing assumptions are an important component of 
the qualitative method selected, is this process explained? 


[J 


[J 


[J 


Context/Setting 








5.1 


Is the whole context for the study adequately described? 


[J 

I 1 


[ 1 

L — 1 


r 1 


5.2 


Is a plan for gaining entry to the setting given? 


r 1 


r 1 
LJ 


r 1 
[J 


5.3 


What contextual difficulties are expected? 


U 


[J 


[J 


5.4 


How will the researcher reduce her/his changing the 
naturalness of the setting? 


[J 


U 


[J 


Sampl 


e 








6.1 


Are the unique issues of sampling in a qualitative study 
adequately addressed? 


u 


[J 


[J 


6.2 


Are the potential characteristics of the sample outlined? 


u 


[J 


[J 
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7. Researcher's Stance 

7.1 How does the researcher explain her/his personal interests 

and biases relating to the study? [J [J [j 

7.2 How will the researcher record her/his involvement and 

reactions as the study proceeds? [j [ j [ j 

7.3 How will the researcher operate as the data 

gathering "instrument"? [J [ J ' [ J 

8. Data Collection 

8.1 Is there a need to use any quantitative methodologies, 
as well as qualitative ones? If so, how will the two 

approaches complement each other? [J [ J [ j 

8.2 Does the researcher demonstrate knowledge of general 
research strategies such as participant-observation, 
interviewing, recording fieldnotes, and conducting 

ongoing analysis? [j [j [j 

8.3 Are the strategies congruent with the purpose of the 
study, the research question, and the type of 

qualitative research selected? [J [ j [ j 

8.4 If so, does the researcher demonstrate skills in their use? U U U 

8.5 How will the four general criteria for trustworthiness 
be applied: i.e., credibility, transferability, 

dependability, confirmability? U U [ J 
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Data Analysis amd Interpretation 

9.1 Does the researcher outline a plan for keeping data 

organized and retrievable? [J [ j [j 

9.2 How will the researcher apply inductive approaches? [ J [ J [J 

9.3 Are tentative theoretical frameworks for analysis identified? U U [J 

9.4 What triangulation measures are planned, i.e., use of 
multiple data sources and analysis methods to increase 

the validity of the findings? [ J [J [ j 

9.5 How will the study^s findings be checked for validity 

with the participants? U U U 

9.6 If the theoretical framework is to be derived directly 
from the data, does the researcher show how this will 

be accomplished? [ j [j [j 

9.7 Does the reseai*cher indicate the expected limitations of 

the proposed analysis methods? [j [j [j 
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10. Participants 

10.1 Does the researcher demonstrate an understanding of the 

measures necessary for the protection of participants? [J [J [J 

10.2 If a written contract or letter of informed consent 
for participants is called for, are examples included 

in the proposal? U U U 

10.3 Is an ethical review by peers of the whole study built 

into the process? [J [Jj [J 

10.4 How will participants and researcher interact in data 

collection and analysis? U U U 

10.5 Will the participants be invited to accept a summary/full 

report of the study? [J [J [J 

11. Time Frame 

11.1 Is there provided a reasonably detailed schedule? [J [J [J 

11.2 Is the schedule feasible? [J [J [j 

12. Presentation of data and findings 

12.1 How will the findings be organized - e.g., as role 
analyses, or critical incidents, or themes, or interaction 

networks, or a natural history? [J [J [J 

12.2 Who is the intended audience? e.g., do they need/want 
more description (academics) or more analyses and 

summaries (policy makers)? [J [J [J 



5 



ERLC 



21 



